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COPYRIGHT NOTICE 
A portion of the disclosure of this patent document contains material' which is 
subject to copyright protection. The copyright owner has no objection to the facsimile 
10 reproduction by anyone of the patent document or the patent disclosure as it appears in 
the Patent and Trademark Office patent file or records, but otherwise reserves all 
copyright rights whatsoever. 

COMPUTER PROGRAM LISTING APPENDIX 
A Computer Program Listing Appendix A containing computer program listings is 
1 5 included with this application. T lie c;£cl osurc of the Computer Program Listing 
Appendix A is hereby incorporated by reference. 

B ACKGROUND OF THE INVENTION 
The present inv ention relates generally to information processing and, more 
particularly, to an online system providing methodology for improving online access to 

20 digital media and related information, 

A variety of digital image, digital video and digital audio products are currently 
available to consumers. Regardless of how digital media is recorded, at some later point 
in time, the information must become available to other devices ~ that is, become 
available to a larger network of digital devices, so that such information may be displayed 

25 on a screen, printed to hard copy, stored, or shared with other people. Today, a large 

number of Web sites exist on the Internet with server computers hav ing the capability to 
organize and display images, video, audio, and other types of media and digital objects. 
In a complementary manner, a multitude of different client devices exist with sufficient 
graphics capabilities for potentially viewing (and/or listening- to) this media. For instance, 

3 0 such client devices range from desktop- or laptop- computers running Web browser 
software to handheld devices (e.g., personal digital assistants rumiing under Palm or 
Windows CE), all connected to the Internet over TCP/IP, each with the capability of 
displaying information. 
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More recently, anewer class of devices has been introduced with wireless data 
capabilities as well as display capabilities. These devices connect to the Internet typically 
over WAP (Wireless Application Protocol). WAP is a communication protocol, not 
unlike TCP/IP, that was developed by a cdnsortiurn of wifeless companies, including 
5 Motorola, Ericsson, and Nokia, for transmitting data over wireless networks. For a 

description of WAP, see e.g., Maim, S., The Wireless Application Protocol, Dr. Dobb's 
Journal, pp. 56-66, October 1999. 

All told, a plethora of graphics-equipped devices exist today that may be 
connected to the Internet, through both wireless (e.g., 9600 baud) and wireline 

10 connections (e.g., 56k baud, DSL, and cable modem). These devices are capable of 

displaying graphics, including digital images. Recent advances in handheld computing 
and wireless technologies have enabled manufacturers to embed or include Web browsers 
in a wide array of devices, including palm-type organizers, wir eless phones, and two-way 
pagers. While all of these Web-enabled clients are capable of at least rudimentary display 

1 5 of text, some also support various multimedia content such as images, video, and sound. 

Faster wireless Internet access will drive development of client devices capable of playing 
larger, richer media formats. 

However, a fundamental problem exists with current approaches to displaying 
images and other digital media on many of these devices. Some devices such as personal 

20 or laptop computers are capable of receiving and rendering a number of different types of 
media, including, digital images,, streaming audio, and streaming video. However, otter 
devices have much more limited capabilities to render digital media, hi addition, 
different devices often support different formats and communication transport protocols. 
Consider, for instance, the example of a Palm handheld device. Suppose a user 

25 has a "true-color" (e.g., 32-bit depth) 1024 by 768 digital photograph of his or her family 
on the Web. If the user connects to the Internet using the Palm device, he or she cannot 
display the photograph because the Palm device may only support four-level or sixteen- 
level grayscale. Even if the image could be displayed, the transmission time for 
downloading the image to the Palm device would be impractical. Still further, even if the 

30 image- could be downloaded, the display for the Palm may be physically too small to 
render the image in a manner that is acceptable to the user. 

This problem is not limited to just image data but also applies to other types of 
digital objects. Many Internet sites display multi-colored images, streaming video, 
streaming audio, and other content that is targeted, primarily at users with desktop and 
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laptop computer devices and Web browser software, A desktop or laptop computer has 
significant processing, storage, and display resources and is capable of rendering large 
images and video files in multiple colors and formats. On the other hand, a smaller 
device such as a cellular phone or a personal digital assistant ("PDA") typically does not 
5 have the necessary capabilities to display colors or to handle media in particular formats. 
For example, a cellular phone, which typically has a small screen for display of messages, 
does not have the software or display capabilities to render a JPEG image. Currently, a 
user with a PDA or cellphone is typically unable to access much of the information 
available on many Internet sites. 

1 0 Certain content providers that are focused primarily on cellular telephone and 

PDA users do offer Internet sites that display media appropriate for these users. These 
sites include images with lower resolution and fewer colors in formats supported by these 
types of devices. However, even content providers focused on cellular telephone and 
handheld device users have problems supporting the wide number of devices currently in 

3 5 use. Numerous different types of cellular telephones and handheld devices are in use, 

each having different specifications and capabilities, These devices have different screen 
sizes, resolution, and color capabi lities. Thus, even if a content provider is focused oil 
cellular telephone users, that content provider often must create images and other media 
offerings geared towards the least capable devices in order to serve the broadest number 

20 ofusers. 

Another particular problem in the delivery of media to wireless users is limited 
bandwidth. In addition to the image display and audio output limitations of many 
wireless devices, the available bandwidth also hmits the ability of these devices So access 
and use richer media formats. Thus even for devices with better audio and video 
25 capabilities, bandwidth considerations may still significantly constrain the types of media 
that may be supplied to these devices. Even if a user's wireless device is capable of 
displaying a high-resolution image, he or she may request a lower resolution image 
because of the time required to download the image given the limited available 
bandwidth, 

30 In addition to these problems stemming from limited bandwidth and display 

resources available to many wireless devices, another problem is that present-day Internet 
services do not attempt to understand the capabilities of particular connected devices. 
Moreover, even if such Internet services understand some of the capabilities of a 
particular client devices, present-day servers are not designed to act on that information 
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and transmit information only in the format that is meaningful for such client device. As 
more and more classes of network-connected devices come to market, this problem can 
be expected to grow as each device has its: own particular characteristics. 

What is needed is a solution that combines oii-the^fly media reformatting with 
5 advanced client detection to enable the delivery of the appropriate and best possible 

incarnation of a provider's mediate every connected client device. Hie present invention 
fulfills this and other needs. 
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GLOSSARY 

The following definitions are offered for purposes of illustration, not limitation, in 
order to assist with understanding the discussion that follows. 

Apache: Apache is a public domain Web server developed by a group of volunteer 
5 programmers called the Apache Group, The initial version of the Apache server was 
developed in 1995 based on the httpd Web server developed at the National Center for 
Supercomputing Applic ations, University of Illinois, Urbana-Champai gn . Additional 
information on the Apache Web server and copies- of the source code for this Web server 
are currently available on the Internet at http://www.apache.org. 

10 CC/PP: GC/PP is an abbreviation for Composite Capabilities/Preference Profiles, a 
proposed standard being developed by the World Wide Web Consortium (W3C). A 
CC/PP profile is a description of device capabilities and user preferences that can be used 
to guide the adaptation of content presented to that device. The current proposed 
specification is described by "Composite Capability/Preference Profiles (CC/PP): A user 

1 5 side framework for content negotiation", W3C Note, 27 July 1999, available from the 
Worl d Wide Web Consortium (W3C. A . copy of the proposed standard can be currently 
found an me Internet &thttp:J/www,w$.0rgnWNQTE-CCPP/, 

HTML; HTML stands for HyperText Markup Language. Every HTML document 
requires certain standard HTML tags in order to be correctly interpreted by Web 

20 browsers. Each document consists of head and body text. The head contains the title, 
and the body contains the actual text that is made up of paragraphs, lists, and other 
elements. Browsers expect specific information because they are programmed according 
to HTML and SGML specifications. Further description of HTML documents is 
available in the technical and trade literature; see e.g„ Ray Duncan, Power Programming: 

25 An HTML Primer, PC Magazine, June 13, 1995. 

HTTP: HTTP stands for HyperText Transfer Protocol, which is the underlying 
communication protocol used by the World Wide Web on the Internet. HTTP defines 
bow messages are formatted and transmitted, and what actions Web servers' and browsers 
should take in response to various- commands. For example,, when a. user enters a URL in 
3D his or her browser, this actually sends an HTTP carmnand to the Web server directing it 
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to fetch arid transmit the requested Web page. Further description, of HTTP is available in 
RFC 261 6: Hypertext Transfer Protocol - HTTP/1 J.. RFC 2616 is available from the 
World Wide Web Consortium (W3C), and. is currently available via the Internet at 
http://www.w3.org/Protocok/: Additional description of HTTP is available in the 
5 technical and trade literature; . see e.g. a William Stalhngs, The Backbone of the Web, 
BYTE., October 1996. 

JPEG: Full-size digital images are maintained in a Joint Photographic Experts Group or 
JPEG format. See e.g., Nelson, M. et al., Tfie Data Compression Book, Second Edition, 
Chapter 1 1 : Lossy Graphics Compression (particularly at pp. 326-330), M&T Books, 
10 1996. Also see e.g., JPEG-iike Image Compression (Parte 1 and 2), Dr. Dobb's Journal, 
July 1995 and August 1995 respectively (available on CD ROM as Dr. Dobb 's/CD 
Release 6 from Dr. Dobb's Journal of San Mateo, CA>. 

SMTP: SMTP stands for Simple Mail Transfer Protocol, a protocol for sending e-mail 
messages between servers. Most c-mail systems that send mail over the Internet use 
1 5 SMTP to send messages from one server to another; the messages can then be retrieved 
' with an e-mail Client using either POP or IMAP. In addition, SMTP is generally used to 
send messages from a mail client to a mail server. 

TCP: TCP stands for Transmission Control Protocol. TCP is one of the main protocols 
in TCP/IP net works. Whereas the IP protocol deals only with packets, TCP enables two 
20 hosts to establish a connection and exchange streams of data. TCP guarantees delivery of 
data and also guarantees that packets will be delivered in the same order in which they 
were sent. For an introduction to TCP, see, e.g., RFC 793. 

TCP/IP: Stands for Transmission Control Protocol/Internet Protocol, the suite of 
communications protocols used to connect hosts on the Internet. TCP/TP uses several 
25 protocols, the two main ones being TCP and IP, TCP/TP is- built into the UNIX operating 
system and is used by the Internet, making it the de facto standard for transmitting data 
over networks. For an introduction to TCP/IP, see e.g., RFC 1180: A TCP/IP Tutorial 
A copy of RFC 1 1 80 is currently available atftp://ftp.istedit/i?i-notes/rfcllS(),ixt. 

UAProf: UAProf or WAG UAProf refers to the proposed Wireless Access Group User 
30 Agent Profile Specification about how devices such as cellular phones should describe 
their capabilities to servers. The current proposed specification is described as "WAG 
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UAProf ' (Wireless Application Group User Agent Profile Specification), Wireless 
Application Protocol Forum, Ltd., Proposed Version 30-May-2OOl, available from the 
WAP Forum. A copy of the specification can currently be found on the Internet at 
littp://\mr\vLwapforim.org/mWdocuments/WAP^ 

5 URL: Abbreviation of Uniform Resource Locator, the global address of documents and 
o ther resources on the World Wide Web. The first pari of the address indicates what 
protocol to use, and the second part specifies the IP address or the domain name where 
the resource is located. 

WAP: WAP stands for Wireless Application Protocol, which is a communication 
10 protocol, not unlike TCP/IP- that was developed, by a consortium of wireless companies, 

including Motorola, Ericsson, and Nokia, for transmitting data over wireless networks. 

For a description of WAP, see e.g., Mann, S., The Wireless Application Protocol, Dr. 

Dobb's Journal, pp. 56-66, October 1999. More particularly, WAP includes various 

equivalents of existing Internet protocols. For instance, WML is a WAP version of the 
1 5 HTML protocol. Other examples include a WAP browser that is a WAP equivalent of an 

HTML browser and a W AP gateway (on the server side) that is a WAP equivalent of an 

HTTP server. At the WAP gateway, HTTP is translated to/from WAP. 

XML: Short for Extensible Markup Language, a specification developed by the W3C. 
XML is a pared-down version of SGML, designed especially for Web documents. It 

20 allows designers to create their own customized tags, enabling the definition, 

transmission, validation, and interpretation of data between applications and between 
organizations. For further description of XML, see, e.g., Extensible Markup Language 
(XML) 1,0 specification which is available from the World Wide Web Consortium 
(www.w3.org). The specification is also currently available on the Internet at 

25 hitp://www.w3.org/m'REC-xml. 
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SUMMARY OF THE INVENTION 
The present invention prorates an online system incorporathig improved 
methodologies enabling content providers- to effectively serve the widest array of clients. 
It combines on-the-fiy media refDrniattmg with advanced client detection capabilities to 
5 enable the delivery of the appropriate and best-possible incarnation of a provider's media 
content to every connected client device. 

The online media delivery system of the present invention (commercially 
embodied hi eSwttch™-i?rand -media delivery system) receives requests from client 
devices for media documents or objects, determines the media output capabilities of the 

10 device making the request and serves a transformed version of the media object 

appropriate for the requesting client device. The system includes, a cheat capabilities 
module (CCM) and a media transformation module (MTM). These two modules; 
cooperate to identify a client from an HTTP request, determine its media output 
capabilities, and reformat the source media according to those capabilities. The system 

] 5 also includes a data store containing information on the capabilities of various client 
devices, an (optional) front-side cache for storing transformed media content, and a 
backside cache for local storage of original items of content. 

The system provides access to media content for multiple disparate client devices 
- that is, to target devices of varying hardware and software capabilities. The system 

20 enables delivery of appropriate media content to practically any device with connectivity 
capability. The target devices may include both wireless devices (e.g., cellular phone, 
handheld personal digital assistant (PDA), and pager) as well as wireline devices (e.g., 
desktop computer, laptop computer, and videophone). 

The improved methodology of the system in providing media content appropriate 

25 to a particular device can be sitmmarized as follows. Initially, the URLs of particular 
full-format multimedia objects on an Internet Web site are modified to be served by the 
system of the present invention. This is accomplished by modifying the HTM L pages on 
the Web site to replace the URLs of these lull-format multimedia objects, with URLs 
which point to the server on which the media delivery system is installed and contains the 

30 path to the. media objects. The system acts as an HTTP proxy to those original objects,, 
intercepting requests for the original content and serving a transformed version of the 
content applicable to the requesting client. 
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When a client device receives user input (e.g.* a click) invoking the modified URL 
for requesting a particular multimedia document, file-media output capabilities axe 
communicated to the system by the device or are surmised by the system's client 
capabilities module. If the device is commumcating its capabilities, it does so during the 
5 initiation or during every inlerdetion. Alternatively, the device's capabilities are 

previously stored in the system' s data store based on prior knowledge of the device. 
Based on the information communicated to or surmised by the system, an information 
record, is created describing the target device's capabilities. This infomiation indicates to 
the system what •transformation (if any) is required for translating the original item of 

1 0 media content into a format suitable for the target device. 

After the appropriate media format required by the device is determined, the client 
capabilities module (optionally) checks the front-side cache to see whether the cache 
already stores a version of the object that has been translated into a format suitable for 
this particular target device. If the object (transformed for the target device) already 

1 5 exists in the front-side cache, then the client capabilities module may simply return that 
object to the client device. However, if an appropriate transformed object is not in the 
cache, then the client capabilities module proceeds to pass the object identifier and the 
client, device parameters on to the media transformation module. 

The media transformation module receives requests for a particular item of media 

20 content in a particular format from the client capabilities module. The media 

transformation module obtains the original media object, transforms the object from its 
original format into the format that is desired for the target device (based on die specified 
target device capabilities), and returns the converted object to the client device. The 
media transformation module utilizes a backside cache as an optimization to provide 

25 increased efficiency. Media objects are retained in the backside cache to avoid having to 
retrieve frequently or recently requested items in response to each request. Use of this 
backside cache reduces the number of calls, over the network and expedites conversion 
and return of media obj ects to client devices. 
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BRIEF DESCRIPTION OF THE DRAWINGS 
Fig, 1 is a block diagram of a computer- system in which, softwarc-impleniented 
processes of the present invention may be embodied. 

Fig, 2 is a block diagram, of a software system for controlling the operation of the 
5 computer system. 

Fig. 3 is a block diagram illustrating an online media delivery -system of the 
present invention. 

Figs. 4A-B comprise a single flowchart illustrating the detailed method steps of 
the system in determining the capabilities of a target device and transforming and 
1 0 delivering content to such target device in an appropriate format. 

Fig. 5 is a flowchart illustrating the operations of the client capabilities module of 
the present invention in acting as a proxy for incoming HTTP requests from non- 
compliant devices. 
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DETAILED DESCRIPTION OF A PREFERRED EMBODIMENT 
The following description will focus on the presently-preferred embodiment of the 
present invention, which is implemented in a desktop application operating in an hitemet- 
connected environment running under a desktop operating system, such as the 

5 Microsoft® Windows operating system running on an IBM-compatible PC. The present 
invention, however, is not limited to any one particular application or any particular 
environment. Instead, those skilled in the art will find that the system and methods of the 
present invention may be advantageously embodied on a variety of different platforms, 
including Macintosh, Linux, BeOS, Solaris, UNIX, NextStep, FreeBSD, and the like. 

10 Therefore, the description of the exemplary embodiments that follows is for purposes of 
illustration and not limitation, 

I. Computer-based Implementation 

A. Bask system hardware {e.g., for desktop and server computers) 

The present invention may be implemented on a conventional or general-purpose 

1 S computer system, such as an IBM-compatible personal computer (PC) or server 

computer. Fig, 1 is a very general block diagram of an IBM-compatible system 100. As 
shown, system. 100 comprises a central processing unit(s) (CPU) or processor(s) 1.01 
coupled to a random-access memory (RAM) 102, a read-only memory (ROM) 103, a 
keyboard 106, a printer 107, a pointing device 108, a display or video adapter 104 

20 connected to a display device. 1 05, a removable (mass) storage device 1. 1 5 (e.g., floppy 
disk, CD-ROM, CD-R, CD-RW, DVD, or the like), a fixed (mass) storage device 1 16 
(e.g., hard disk), a communication (COMM) port(s) or interface^) 110, a modem 112, 
and a network interface card (NIC) or controller 1 1 1 (e.g., Ethernet). Although not 
shown separately, a real-time system clock is included with the system 100, in a 

25 conventional manner. 

CPU 101 comprises a processor of the Intel Pentium® family of microprocessors. 
However, any other suitable processor may be utilized, for implementing the present 
invention. The CPU 101 communicates with -other components of the system via a bi- 
directional system bus (including any necessary input/output (I/O) controller circuitry and 

30 other "glue" logic). The bus, which includes, address lines for addressing system 

memory, provides data transfer between and among the. various components. Description 
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of Pentium-class microprocessors and their instruction set, bus architecture, and control 
lines; is available from Intel Corporation of Santa Clara, CA. Random-access memory 
1.02 ;serves as the working memory for the CPU 101. In a typical configuration, RAM of 
sixty-four megabytes or more is employed. More of less memory may be used without 
5 departing from the scope of the present invention. The read-only memory (ROM) 1 03 
contains the basic input/output system code (BIOS) - a set of low-level routines in the 
ROM that application programs and the operating systems can use to interact with the 
hardware, including reading characters from the keyboard, outputting characters to 
printers, and so forth. 

10 Mass storage devices 115,116 provide persistent storage on fixed and removable 

media, such as magnetic, optical or magnetic-optical storage systems, flash memory, or 
any other available mass storage technology. The mass storage may be shared on a. 
network, or it may be a dedicated mass storage. As shown in Fig. 1, fixed storage 1 16 
stores a body of program and data for directing operation of the computer system, 

1 5 including an operating system, user application programs, driver and other support files, 
as: well as other data files of all sorts. Typically, the fixed storage 1 16 serves as the main 
hard disk for the system. 

In basic operation, program logic (including that which implements methodology 
of the present invention described below) i s loaded from the removable storage 115 or 

20 fixed storage 1 1 6 into the main (RAM) memory 1 02, for execution by the CPU 1 0 1 . 

During operation of the program logic, the system 100 accepts user input from a keyboard 
106 and pointing device 108, as well as speech-based input from a voice recognition 
system (not shown). The keyboard 106 permits selection of application programs, entry 
of keyboard-based input or data, and selection and manipulation of individual data obj ects 

25 displayed on the screen or display device 105. Likewise, the pointing device 1 08, such as 
a mouse, track ball, pen device, or the like, pennits selection and manipulation of objects 
on the display screen. In this manner, these input devices support manual user input for 
any process running on die system. 

The computer system 100 displays text and/or graphic images and other data on 

30 the display device 105.. The video adapter 104, which is interposed between the-display 
105 and the system's bus, drives the display device 105, The video adapter 104. which 
includes video memory accessible to the CPU 101,. provides circuitry that converts pixel 
data stored in the video memory to a raster signal suitable for use by a cathode ray tube 
(CRT) raster or liquid crystal display (LCD) monitor. A hard copy of the displayed 
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information, or other infonnation withm the system 100, maybe obtained from the printer 
107, or other output device. Printer 107 may include, for instance, an HP LaserJet® 
printer (available from Hewlett-Packard of Palo Alio, CA), for creating hard' copy images 
of output of the system. 

5 The system itself communicates with other -devices (eg., other computers) via the 

network interface card (NIC) 1 1 1 connected to. a network (e.g., Ethernet network, 
Bluetooth wireless netvvork s or the like), and/or modem 112 (e.g., 56Kbaud f ISDN, DSL, 
or cable modem), examples of which are available from 3Com of Santa Clara, CA. The 
system 100 may also communicate with local occasionally-connected devices (e.g., serial 

1 0 cable-linked devices) via the communication (COMM) interface 11 0, which may include 
a RS-232 serial port, a Universal Serial Bus (USB) interface, or the like. Devices that 
will be commonly connected locally to the interlace 1 10 include laptop computers, 
handheld organizers, digital cameras, and the like.. 

IBM-compatible personal computers and server computers are available from a 

1 5 variety of vendors. Representative vendors include Dell Computers of Round Rock, TX, 
Compaq Computers of Houston, TX, and IBM of Armonk, NY. Other suitable computers 
include Apple-compatible computers (e.g., Macintosh), which are available from. Apple 
Computer of Cupertino, CA, and Sun Solaris workstations, which are available from Sun 
Microsystems of Mountain View, CA. 

20 B. Basic system software 

Illustrated in Fig. 2, a computer software system 200 is provided for directing the 
operation of the computer system 100. Software system. 200, which is stored in system 
memory (RAM) 102 and on fixed storage (e.g., hard disk) 116, includes a kernel or 
operating system (OS) 210. The OS 210 manages low-level aspects of computer 

25 operation, including managing execution of processes, memory allocation, file input and 
output (I/O), and device I/O. One or more application programs, such as client 
application software or "programs"' 201 (e.g., 201a, 20lb, 201c, 201d) may be "loaded" 
(i.e., transferred from fixed storage 1 16 into memory 102) for execution by the system 
100. 

30 System 200 includes a graphical user interface (GUI) 215, for receiving user 

commands and data in a graphical (e.g., ^int^rad-clkk^ fashion. These inputs, in. turn, 
may be acted upon by the system 100 in accordance with instructions from operating 
system 210, and/or client application modules) 201. The GUX215 also serves to display 
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.the results .of operation from the OS 210 and appheatioii(s) 201, whereupon the user may 
supply additional inputs or terminatethe session, Typically, the OS 210 operates in 
conjunction with device drivers 220 (e.g. s "VVinsock" driver - Windows' implementation 
■of a TCP/IP stack) and the system BIOS microcode 230 (i.e., ROM-based microcode), 

5 particularly when interfacing with peripheral devices. OS 210 can be provided by a 

conventional operating system, such as Microsoft® Windows 9x, Microsoft© Windows 
NT, Microsoft® Windows 2000, or Microsoft® Windows XP, all available from 
Microsoft Corporation of Redmond, WA, Alternatively, OS 210 can also be an 
alternative operating system, such as the previously-mentioned operating systems. 

1 0 The above-described computer hardware and software are presented for purposes 

of illustrating the basic underlying desktop and server computer components that maybe 
employed for implementing- the present invention. For purposes of discussion, the 
following description will present examples in which it will'be assumed that there exists a 
"server" (e.g., Web server) that communicates with one or more "clients" (e.g., media 

1 5 display devices). The present invention, however, is not limited to any particular 
environment or device configuration. In particular, a client/server distinction is not 
necessary to the invention, but is used to provide a framework for discussion. Instead, the 
present invention may be implemented in any type of system architecture or processing 
environment capable of supporting the methodologies of the present invention presented 

20 in detail below. 

II. Online rendering of media tailored to capabilities of various devices 
A. Introduction 

Today, a great volume of various types of media content is available on a 
multitude of Internet sites. At the same time, a wide range of target devices exist that are 

25 capable of displaying (rendering) media to users. These devices include personal 

computers, laptop computers, personal digital assistants (PDAs), two-way pagers, and 
cellular telephones. However, several problems exist in the delivery of media content to 
these devices. Given the vastly different capabilities of the various target devices in use 
and the different types of images and other media content available on the Internet, 

3 0 content providers currently have problems- delivering media content in a maimer suitable 
for display (or rendering) to the user of a particular target device in a satisfactory manner. 
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One solution for these problems is for an Internet site to display information in a 
number of different, formats,. However, this solution requires an Internet content provider 
to display a large, number of copies of the same media object in order to address the 
various types of devices on the market and their wide range of capabilities. Among the 
5 disadvantages of this approach are that it requires the content provider to create, store, 
and manage multiple pro-rendered versions of the same media in various formats 
depending on the number of devices to be supported. 

The present invention allows a content provider to develop content in one form 
and deliver the content in multiple forms based on the capabilities of the client device: 

10 requesting the content. The present invention includes an online system and methodology 
for providing a client device with media content appropriate for the- media output 
capabilities of the device. The system includes a client capabilities module (CCM) to 
determine the capabilities of connected client devices and a media transformation (or 
transcoding) module (MTM) mat renders the appropriate media on-the-fly and delivers it 

1 5 to the client device in the appropriate format. 

The operations of the present invention can be illustrated by the following 
example of rendering a digital image to a particular client device. In this example, the 
original item of media content on an Internet site is a 24-bit color JPEG image and the 
client device requesting this image is a . Palm PDA that supports 16-level grayscale. The 

20 client device connects wirelessly to the Internet site and invokes the URL for this JPEG 
image. The Internet content provider using the present invention has previously revised 
the Uniform Resource Locator (URL) for this image to refer to the machine on which the 
client capabilities module of the present invention is installed. As a result, this URL 
request is routed to the CCM, which identifies the requested image and the client device 

25 from this request. The CCM determines the capabilities of the client device in an 

intelligent fashion by examining the client request to the server to obtain information 
about the client device and by comparing this information to known device characteristics 
and capabilities stored in its data store. In this case, the CCM recognizes this device as a 
specific type of Palm PDA and looks up die device's capabilities in the system's 

30 database. Based upon this information, the CCM detemines that the JPEG image should 
be supplied to this Palm device in a 16-level grayscale format. 

After the appropriate media format required by the device is determined, the CCM 
(optionally) checks a front-side cache to see whether the cache already stores a version of 
the image in the required format. If an appropriate transformed image is not in the. cache, 
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then the CCM requests the image from the media transfoimation module in a 16-level 
grayscale format suitable for rendering to this type of Palm PDA device. The MTM 
obtains the appropriate image, converts it to the appropriate format and serves it to the 
client device. The system includes intelligence that allows it to optimally translate the 
images (from their original format) into a format suitable for use by the particular target 
device. The overall translation or transformation process is performed in a manner that 
preserves performance and scalability criteria desired for the system. 

B. Overview of media delivery system 

1. Basic architecture 

Fig. 3 illustrates an online environment 300 suitable for implementing the present 
invention, As shown, the environment 300 includes an online media delivery system 320 
connected via the Internet (shown at 3 10) to one or more client devices 3 01 and at least 
one internet site (server) 330. Each of these components will next be described in greater 
detail- 
Ghent device 301 represents one of a variety of target devic es (or "clients") that 
ate capable of connecting over the Internet and accessing online content. For example, 
client devices may include both wireless devices (e.g., cellular phone, handheld PDA 
(personal data assistant), and pager) as well as wireline devices (e.g., desktop computer, 
laptop computer, and videophone). Although a single client device is shown in the figure, 
typically the environment 300 would operate with a multitude of such devices connected. 

The Internet server 330 represents a Web server at which items of media content 
(e.g., audio, video, documents, blob objects, or other items of interest) are stored. During 
operation, the Internet server 330 typically stores a number of different items of media 
content that are to be made available to a wide range of client devices. Actual connection 
'between the Internet server 330 and the online media delivery system 320 may occur over 
the Internet or, optionally, occur via a non-Internet (e.g., WAN) connection as illustrated 
by the dashed connection line in the figure. In either instance,, the Internet server 330 
may be housed at the same site as die online media delivery system 320, or may be 
housed at a remote site, as desired. 

The online m. . ■_' ( functions to detect client device capabilities 

and, based on that determination,, transforms and delivers media content to such devices 
in appropriate formats to the client device 301. As shown, the media delivery system 320 
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includes a client capabilities module (CCM) 322, amedia transformation module (MTM) 
325, and a device capabilities data store. 324. Aa .also shown, the client capabilities 
module 322 is in direct communication with a front-side cache 321 and a CCM log 323; 
the media transformation module 325, similarly, is in. direct communication with backside 
5 cache 327 and. MTM log 326. 

2, Basic operation 

During basic system operation, items containing and/or referencing media content 
(e.g., Web pages) on the Internet server 330 are encoded with a URL that directs clients 
requesting such items to the system 320. The Internet server 330 may also include the 

1 0 original items of media content, which may be any type of content including digital 

images, video, audio, documents, "blob" objects, or the like. Alternatively, original items 
of media content may be stored locally on the system 320 or on another local or remote 
server to which the system 320 is connected. When a request (e.g., HTTP request) for an 
item of content is made by the client 301, the request is routed to the client capabilities 

i 5 module 322 of the media delivery system 320. Responsive to the request received from 
the client device 301, the client capabilities module 322 identifies the (client) device and 
obtains available information about the device's capabilities. Based on this identification, 
the client capabilities module 322 retrieves additional information about the capabilities 
of the client device for displaying or outputting media from the data store 324. 

20 The data store 324 includes media output capabilities of various devices. In the 

currently preferred embodiment, a corresponding device identifier is employed to index 
this information. The capabilities stored in data- store 324 include information regarding 
screen resolution, screen color depth, whether images should be rotated to fit on the 
device's screen display, and other such. information as described in more detail below. 

25 The data store 324 is field upgradable so that as new devices are introduced into the 
market, the profiles of such devices and their capabilities can be added. The client 
capabilities log 323 includes a record of any client devices that could Hot be identified or 
for which capabilities are not available. These log records enable any omitted devices to 
be identified so that information on these devices can be obtained and added to the data 

30 store 324. 

After the capabilities of clieni device 301 have been determined, client capabilities 
module 322 (optionally) looks into the front-side cache 321, which stores previously 
converted content, to see if the object is available in the appropriate format. The front- 



page 17 of 51 



WO 03/0411893 



PCT/l)S02/3eo64 



side cache32i is an (optional) optimization in which previously converted media objects 
are retained for supply in response- to future requests. The front-side cache 321 may be 
implemented/using least-recently used (LRU) technique- to "age out" (i.e., remove) the 
least-recently used items. If the client capabilities module 322 determines that the 
5 appropriate obj ect is not available in the front-side cache 32 1 , it requests the media 

■transformation module 325 to perform an on-the-fly transformation that will supply the 
object. 

The media trmisforniafion module (MTM) 32.5 receives requests for a particular 
item of media content in a particular format from client capabilities module 322. The 

1 0 media transformation module 325 obtains an original copy of the requested media obj ect , 
converts it into the requested format, and returns the converted media object; to the client 
device 301. The backside cache 327 is an optimization to provide increased efficiency; it 
may also be implemented using LRU technique. Original objects retrieved from the 
Internet server 330 (or another source) are retained in the backside cache 327 to avoid 

15 having to retrieve a copy of each requested item in response to each request. Use of this 
backside cache reduces the number of cal ls over the network. It also expedites 
conversion and return of available objects by the media transformation module 325 by 
avoiding the retrieval of large objects (such as high q uality color images) from a remote 
server. 

20 C. Methodology for detecting capabilities of devices and for delivering 

appropriate media objects 

Figs, 4A-B comprise a single flowchart of the detailed method steps of the 
operations of the system in detecting the capabilities of connected client devices and 
delivering media, objects to such devices in appropriate formats. In step 401 , URLs for 

25 multimedia objects in Web pages at an Internet site are modified so that such objects will 
tee served by the media delivery system of the present invention. In the currently 
preferred embodiment, the URLs are prepended with the server name on which the media 
delivery system is installed. For example, if the subscriber's site contained a logo 
normally accessed by the URL: kttp:/Ann^mbscriber,com/img/logo.gif, flic modified 

30 URL would be: http://cswitch.comfmm'.suhscriher.com/m^ 

In step 402, an HTTP 1 request from a client device is routed to the media delivery 
system when a Web page containing these rewritten URLs is opened or the client device 
selects (clicks on) a rewritten URL. In step 403, the client capabilities module (COM) 
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reverses the encoding process pertbrmed in step 401. and determines the Ml. URL to the 
source image. In the currently preferred implementation, this consi sts of removing the 
7eswitch.com" .from the request URL,- 

In step 404 s: the CCM retrieves the client capability configuration from the data 
5 store using the HTTP User- Agent header as a key. The configuration information 

specifies the playback capabilities of the client device, such as display size, color depth, 
audio channels, and so forth. The configuration information may require examination of 
additional HTTP request headers to determine the complete capabilities of the client 
device. Information gathered during this step allows the CCM. to understand exactly what 

1 0 capabilities are supported in the target device. In particular, this information indicates to 
the system what particular transformation operation(s) is requited in order to translate the 
original media object into a format suitable for the target device. 

In step 405, the CCM constructs a URL containing commands specific to the 
media transformation module (MTM). These commands instruct the MTM to transform 

15 the source media document or object to conform to the capabilities of the client device 
that requested the document. This URL points to the MTM server specified in the 
configuration file. The MTM module can be on the same server or a different server than 
the CCM. In (optional) step 406, the CCM consults a front-side cache for an object 
matching the constructed MTM URL. In other words, it looks to see if the front-side 

20 cache already stores a version of the media object that has been translated in a manner 
suitable For this particular target device. 

In the currently preferred embodiment, the URL strings used internally within tire 
system are encoded to serve as an index for particular object in a particular format In 
this manner, an encoded URL string can indicate that a particular document in a particular 

25 format is stored at a particular location. For example the CCM can check for a URL that 
includes a transformed version of logo.gif with the characteristics: size = 100 pixels, color 
depth = 8, and color - false. If the document or object (translated for Che target device) 
already exists in the front-side cache, the CCM may simply return the document to the 
target device. However, if a matching item is not found hi the cache, then the method 

3 0 continues as described in step 407. 

In step 407, the CCM proxies the original client request, replacing the URL sent 
by the client with the reconstructed URL created by the CCM. This process is completely 
■transparent to the client: the client device making; the request is not informed or aware 
that the request has been passed on to the MTM. Rather, this transfer is a back-end 
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process in which the CCM forwards the request made by the client device for fulfillment 
by the.MTM. In step 408, the MTM receives die constmcted URL and makes an HTTP 
request through a backside-caelxnig server For the original media object. If the object is 
present in the backside cache, it is served from local disk. If not, the caching server 
5 requests the obj ect from the Internet site identified in the original URL and caches it for 
future use. The task of the MTM, at this point, is to transform the media object from its 
original format into the format that is desired for the target device (based on target device 
capabilities). Tn step 409, the MTM performs the media transformation as specified in the 
reconstructed URL that it received. Once the MTM has carried out this task, in step 410 
1 0 the newly-transformed version of the media object is returned to the client device and 
(optionally) is also copied into the front-side cache. 

D. Use of system to determine and provide information on client capabilities 
In addition to serving the role described above, the present invention may also be 
used to determine the capabilities of client devices and supply this client capabilities 

15 information to other systems or devioos. For further description about how devices such 
as cellular phones should describe their capabilities to servers, see, e.g., WAG UAProf 
(Wireless Application Group User Agent Profile Specification), Wireless Application 
Protocol Forum, Ltd., Proposed Version 30-May-2001, available from the WAP Forum. 
A copy of the specification can currently be found on the Internet at 

20 http://wwwLwapfonim.org/tech/do^ A 

similar specification is described by Composite Capability/Preference Profiles (CC/PP): 
A user side framework for content negotiation, W3C Note, 27 July 1999, available from 
the World Wide Web Consortium. (W3C). A copy of the specification can be currently 
found on the Internet at http://mwJw3.Qrg/rR/NOTE-CCPP/. 

25 One or both of these proposed standards may be supported by new devices and 

device software in the future, but current support for such standards is very limited. Until 
the device support of these standards is universal, server applications will not be able to 
take advantage of the standardized device information. This problem can be addressed by 
configuring the system of the present invention to act as a proxy for incoming HTTP 

30 requests from non-compliant devices. When a request is forwarded to the media delivery 
system with partial or no UAProf or CC/PP information, the client capabilities module 
looks up the required data and attaches this information to the request before forwarding 
the request on to its eventual destination. This: enables the system to act as a bridge 
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'between the non-compliant client device and those Internet and WAP sites that require 
compliance with UAProf, CC/PP, or other comparable standards. 

Fig. 5 is a flowchart illustrating the operations of the client capabilities module of 
the media delivery system in acting as a proxy for incoming HTTP requests from noii- 

5 compliant devices, to step 501, an HTTP request from a non-compliant client device is 
forwarded from an Internet or WAP site to the system. For these purposes a "non- 
compliant" client device is one that is not in compliance with UAProf, CC/PP, or similar 
stamlfai ds requiring such device to identify its capabilities. 

In step 502, the system's client capabilities module (CCM) retrieves the client 

1 0 capability configuration from the data store using the HTTP User- Agent header as a key. 
The configuration information specifies the capabilities of the client device, such as 
display size, color depth, audio channels, and so forth. The configuration information 
may require examination of additional HTTP request headers to determine the complete 
capabilities of the client device. Information gathered during this step allows the CCM to 

1 5 understand exactly what capabilities are supported in the target device. 

In step 503, the CCM supplements the request made by the particular client device 
with information regarding the specific capabilities of such client device as illustrated 
below. In step 504, the CCM returns the supplemented request including details on the 
client capabilities to the destination specified in such client request. 

20 The following is an example of how the system can be used to append device 

capabilities information to a request. An example of an incoming request forwarded to 
the system is as follows: 



GET /index. wml HTTP/ 1.1 
25 Host : www . light surf . com 

Accept-Charset : ISO- 8 85 S 1 
Accept -Language:- en 

x-up-subno s pegli_pegli-nt4 .of f ice . lightsurf.com 

,x-upf ax- accepts i none 
30 x- uc -uplink : none 

x-up-deveap-smartdialing: 1 

■' n] i a, -.i i ' Irfjta: 1 

x-up-devcap-iscclor : 0 

jc-up-devcap-iromed-alert: 1 
35 x-up-devcap-numsoftJcsys; 2 

x-tp-devcap-screenpixels-: 171,10 8 

x - up - devcap -ms 1 y. e ; 8,18 

User-Agent: UP -Browser/ 3 . 1-TJPG1 UP, Link/3:. 2 
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As shown, the incoming information reports device capabilities including, for example, 
color support ("0" or none, for the above device) and screen pixels (171 by 108 pixels for 
the above device). 

Following receipt of the above: request, the client capabilities module determines 
5 the capabilities of the particular plient device in the manner described, above. The CCM 
then attaches this information to the request and forwards the supplemented request on to 
its eventual destination. A sample request showing the information appended by the 
CCM is as follows; 



10 get /index. wml http/1.1 
Host: www . .1 i ght surf .com 
accept - Charset s I SO - 8 S 59 - 1 
Accept -Language : en 

User-Agent r UP . Browser/3 . 1-UPG1 UP. Link/? .2 
15 x-wap-profile: http: //www. ©switch, con/prof ile3/QA3F3G2B,xnil 

In this instance, "0A3F362B.xml" is a generated document containing either the UAProf 
or CC/PP profile information. A sample UAProf file for this request is as follows: 

20 <?-xml version=" i . 0"/> 

<RDF xmlns= "http t //www. w3 . org/ 1999/02 / 22 -rdf ~syntax-nB# " 

xmlns : rdf =" http : / /www . w3 . org/ 1-9-9 9 / 0 2 / 2 2 - rdf - syntax-ne#" 

xmlns sprf- "http : //www . wapf orra . qrg/prof 1 les/UAPROF/ ceppechema-a 0 0 1 04 3- 0 # "> 
25 <rdf : Description id="WAP-enabled cellular phone" > 

<rdf : type 

res our ce = "http-: //www . wap form . org /prof i 1 es / UAPROF / c cps c hema - 
. 20010430 #Hardware 
Platform" /> 

30 <prf : sereenSlse>171xl08</prf :-Scr««nSi-ze> 

<prf ■ :ColorCapable>No</prf :ColorCapable> 
<prf : NnmberOf Sof tKeys >2 </prf :HamberOf SbffcKeyB> 
<prf :ScrecnSiseChar>8xl8</pr£:3creenSizeChar> 

35 </ rdf : Description 

</KDF> 

The generated document is populated with information from the device's HTTP request 
and the data store or knowledgebase maintained by the media delivery system. Because 
40 the system mai ntains a knowledgebase of client device characteristics, it is much more 
capable of creating a complete UAProf or CC/PP document than a server which simply 
transcodes the information in. the HTTP .headers. 
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E, Detailed methods of operation of CCM and MTM modules 

The client capabilities module (CCM) identifies a client device from an HTTP 
request. The CCM uses information supplied in the request, together with 'media output 

5 capability information stored in the data store, to determine the media output capabilities 
of a particular client device. This enables the CCM to determine the optimal transmission 
size and playback format for the particular type of client device requesting the item of 
interest (e.g., media object). For example, an HTTP browser may indicate the browser 
name (e.g., "Netscape Navigator" or "Microsoft Internet Explorer"), the browser version, 

1 0 and the graphic types it supports. This information is helpful in instances where graphic 
support is limited, such as a browser running on a set-top box having very limited graphic 
support (e.g., JPEG support only). In the case that the target device is not able to indicate 
its capabilities, it will at least indicate its device class, such as a Palm handheld device, a 
set-top box, a phone with a WAP browser, or the like. 

15 Based on the device class, the CCM obtains information about the capabilities of 

the device from the device capabilities data store. For example, the device class may be a 
Palm handheld device of a particular model. Based on this information, the CCM may 
discern, by looking up the device class in the knowledgebase maintained in the data store, 
the capabilities of the device, such as display capability (e.g., 16-color). device memory 

20 (e.g., 4-8 MB), and display size (e.g., 300 x 500). 

The CCM also includes methods to log unidentified clients in the CCM log and to 
provide notifications at regular intervals to ensure that the knowledgebase is as up-to-date 
as possible, As new client devices are introduced, the configuration information in the 
knowledgebase can be updated to add information on these new client devices. The CCM 

25 supports either a "push" or a "push/pull" scheme for updating its configuration files. The 
"push" scheme consists of a secure HTTP post request or SMTP message sent to the data 
store containing lite replacement configuration file. The 1, pusfa/puU" scheme consists of 
sending an HTTP get request or SMTP message to the data store which causes the server 
to schedule an update of the local configuration files. 

30 The CCM relies primarily on HTTP headers,, particularly the User- Agent header, 

to identify clients. The CCM also examines information in the protocol layer below 
HTTP (for example, the origin of the request's BP packets). Once the device has been 
identified, the CCM consults -a hierarchical configuration file (in the data store) which 
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supplies default values for the device's capabilities- and specifies additional headers, such 
.as the Accept header, which can also 'be used to determine the proper output format for 
media content. Once the device's capabilities have been determined, the CCM constructs 
a request to the MTM for the specified media document including the proper reformatting 

5 information. The CCM then forwards the client connection to the MTM with the 
reconstructed request. 

In the currently preferred embodiment, the CCM may be implemented as an 
Apache module with access to the full HTTP request made by the client. The information 
contained in that request is used to identify the client device making the request through a 

1 0 series of queries against a configuration file. The HTTP User-Agent header is the 

primary indicator of the requesting client. While User- Agent is an optional header in 
both HTTP/1.0 and HTTP/1.1, in practice all Web client software send some identifying 
information in that header on each request. Many clients also send custom headers 
describing the physical capabilities of their devices. For example, the UP browser 

15 (Unwired Planet browser supplied by Openwave Systems, Inc. of Redwood City, CA), 
which is a W AP browser used in mobile phones, sends out "non-standard" heads in the 
request. "Non-standard" in this context means that such headers are not covered by the 
HTTP specification. An example of one of these headers is as follows: 



20 user-Agent: UP. Browser/3 .04 -SC02 UP .Link/ 2.2.3.8 

x- up - dssrc ap - chars et : US-ASCII 

x-up -devcap-iramed- alert : 1 

x-up-devcap-raax-pdu : 1472 

x-up-deveap-msize: 8,10 
25 x - up - devc sip - numsof t keys : 2 

x up cevcap-screenchara : 15,5 

x-up-devcap-screenpixel B • 120,50 

x-up-devcap-smartdialing: 1 

x-up-deveap-sof tkeysise : 7 
30 x-up-f ax-accepts : none 

x-up-f ax-limit: 0 

x-up - subno : RaOysSrSj ~ARAs01_up2 .upl . sprintpes . com 
x-up -uplink ! up2 . upl . sprintpca . com 



35 The "x-up-devcap-screenpixels" portion of the above- header specifically indicates bow- 
many pixels in width and height (120 x 50) the client device is capable of displaying. The 
header shown in the abov e example contains several details about the capabilities of this 
particular client device. However, in many cases headers do not include all of these 
details, and. thus the CCM must refer to information stored in the data store to obtain 
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device characteristics. However, the CCMuses information in the headers like "x-up- 
devcap-screenpixels" portion of the above header whenever possible. 

In the currently preferred implementation, the CCM configuration file (or 
knowledgebase) in the data store is written in XML to take advantage of that language's 
5 hierarchical features. The file consists- of a series of <user-agent> entries. An example 
<uscr-agent> entry might appear as follows (line numbers are for reference only): 



1 <-jser- agent header = 'User -Agent 1 pat tftrn=> UP. Browser' >> 

2 <devi ce-claas > wap «;/ device - a 1 a s e > 

3 <content-type pattern= ' A image/ ' > 

4 capability naiiie=' color-depth' default-' 8 ' /> 

5 ^capability nair.es ' display-height ' default** ' 100 * > 

6 <header name='x-up-devcap-screenpixels ' 
pattern» : ' < \d+) , \d+ 1 / > 

7 </capability> 

8 <capability name-' diaplay- width' default- ' 100 ' > 

9 <header name- 'x-up-devcap-screeiipi.xssls ' 
pattern- '\d+, (\d+) •/> 

10 < /capability* 

11 .^capability name" ' output-format 1 defaults- 1 image/bmp ' /> 

12 < /content- type > 

13 </user-agen.t> 

In general, tag attribute naming patterns indicate that the values provided to those 
attributes will be treated as regular expressions. In the case where information must be 
extracted from the regular expression, standard "match remember" syntax (parentheses) is 
used. For example, on line 9 above/the pattern attribute matches the string "120, 50" and 
indicates that the substring "50" is to be used to set the enclosing capability value. 

When the CCM receives a request, it runs through the configuration file, checking 
each <user-agent> tag in turn by comparing the value of the HTTP header specified by 
the header attribute to the string, specified in the value attribute. In the majority of cases, 
the <user-agem> tag is matched against the HTTP User-Agent header, and matching the 
tag against the HTTP User-Agent header is the default behavior if the header attribute is 
omitted. Each <user-agent> block is processed in the order it appears in the configuration 
file, so <user-agent> tags with more restrictive value attributes appear before <uscr- 
agent> tags with icss-rcstrictivc value attributes. For example, a <user~agent> tag which 
reads value='UP . Browser/3. 1 ' is placed before a <uscr-agent> tag which reads 
value=UP. Browser'. The <device class> element is used to separate the different clients 
into arbitrary groupings based on their capabilities. Some examples of <deviee-elasa> 
values might be "WAP", "i-mode", "HDML", and "j-phone." 
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Once a matching <user-ageiit> tag is found, the CCM determines the MIME type 
.of the media document being requested, generally by issuing an HTTP head request to 
the document source. Once the MIME- type is known, the CCM looks up the appropriate 
<content»type> block inside the <user-agent> block, hi the example above, any request 
5 for MIME types that start with "image/"' -will match the regular expression defined in the 
pattern, attribute of the first <content-type> tag. An actual configuration file may have 
separate blocks for each supported media type or subtype. 

Client capabilities are defined inside the <content-type> tag by one or more 
•-'capability?- tags. Each media type may require different capabilities. To properly 

1 0 display images, one needs to know display width, height, and color depth. To supply 
appropriate audio streams, one needs to know bandwidth and whether the device is 
capable, of playing multiple channels. In the simplest case, the configuration file provides 
previously-stored values for each capability through the mandatory default attribute. 
Lines 4 and 1 1 of the above example illustrate this ease, setting the color depth and output 

1 5 format for all URBrowser user agents to 8 bits per pixel and image/bmp, respectively. As 
described above, some user agents provide additional information to the server in the 
' form of non-standard HTTP headers. The <header> tag can be used inside of a 
<eapability> tag to instruct the CCM to parse these headers and set the device capabilities 
depending on the header values. In the example, line 6 indicates that the non-standard "x- 

20 up-devcap-screenpixels" header should be used, if present, to set the devic e display width 
by applying the regular expression provided in the pattern attribute to that header's value. 
Parentheses are used to isolate a part of the header value to assign to the capability. 

Although not specifically shown above, the underlying communication transport 
may also be inferred from the class or type of device. For example, if the target device is 

25 a cellular phone, the system may inter that the underlying communication transport is 

wireless. As. another example, if the target device is a pager that is communicating using 
WAP, the system may infer that -the. target device uses wireless communication with 
limited bandwidth (as compared' to a cellular phone). Based on the device class and the 
incoming request, the system may usually discern whether the communication transport is 

30 wireless or wireline. Moreover, very few devices have both wireless and wireline 

capability. Typical wireline connections include Tl, DSL, cable modem and dial-up 
connections. On the wireless side, typical connections include 9600-baud 
circui t- switched data call, 9600-baud packet-switched data call, or the newer 64K baud 
GPR call. 
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The client capabilities module is also responsible for verifying the source of the 
original content so that the system oan only be used to reformat content of authorized 
participating sites and not of third parties. Security is enforced by only activating the 
CCM in response to specific URLs containing the name of a designated server for which 
5 content is to be transformed. 

The CCM uses the information it derives about the capabilities of a particular 
client device to construct a request to the media transformation module for the requested 
media document. This CCM forwards this constructed request, including the proper 
reformatting information and the client connection to the MTM. The client capabilities 
1 0 module (opti onally) implements a front-side cache for transfortned media documents. 

This front-side cache is consulted for transformed document meeting the criteria of the 
constructed request before the request is forwarded to the MTM, The purpose of this 
front-side cache is to minimize the load on the MTM module. 

2, Media transformation module 

1 5 The media transformation (or transcoding) module (MTM) accepts HTTP requests 

for media documents, which contain formatting instructions as request parameters, and 
reformats die media according to those instructions. The MTM may specialize in a single 
media type, such as image or video or may support multiple types of media. In basic 
operation, the media transformation module supports image reformatting by: converting 

20 images both to and from the following MIME types: image/jpg, image/bmp, image/gif, 
image/tiff, image/wbmp, image/iff, image/pox, and image/png; decreasing imago 
dimensions and increasing image dimensions; supporting rotation of images to conform 
to the aspect ratio of the client display; and supporting decreasing image color depth and 
increasing of image color depth. The MTM supports audio reformatting by: converting 

25 audio files and streams both to and from the following MIME types: audio/aiff, audio/au, 
audio/mpeg, audio/wav; and decreasing audio bit rate. 

The MTM supports video reformatting by converting video files and streams both 
to and from the following MIME types: video/mpeg, video/quicktime, video/x-msvideo 
(AVI), video/x-ms asf, video/rm, and vidco/mjpeg. The MTM can also support 

3 0 reformatting of additional multimedia content types and streams as defined by RFC 2 046, 
Multipurpose Internet Mail Extensions (MIME). A copy of RFC 2046 is currently 
available on the Internet at http://miWAetf.Qrg/rfdrfc2Q46.txt. 
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An example of the operation of the media transfonnation module demonstrating 
its operation is set forth below, in this example, the MTM receives is translating a JPEG 
image arid.receives as input the following: 

Dimensions of output (width and height); 

Type of output device; 

Color space (e.g., RGB or Grayscale); 

Color palette; (e.g., True color or indexed); and 

Compression technique. 

From these inputs, the MTM may output a picture in the specified output format at the 
specified size. Note that' some devices may be characterized differently than their native 
characteristics. For example a device with a 1 6-cobr LCD display may prefer to be 
characterized as a true-color device. It is then the responsibility of the device to convert 
the true-color mode image transmitted by the MTM to its internal 16-color mode using 
internal software/hardware, Any suitable compression scheme may be employed, 
including proprietary or non-proprietary schemes. Examples include JPEG,. JBIG, GIF, 
or the like. See, e.g., JPEG-like Image Compression (Parts 1 and 2), Dr. Dobb's Journal, 
July 1995 and August 1995 respectively (available on CD ROM as Dr. Dobb 's/CD 
Release 6 from Dr. Dobb's Journal of San Mateo, CA). 

The specific operations of the MTM in translating the above-mentioned JPEG' 
image are as follows. First, the input picture is decompressed to generate a bitmap in the 
color space that was employed. For example, Clikpix uses GUV color space; J PEG 
supports multiple color space, including YUV and RGB. GUV color space is described 
in commonly-owned application serial number 09/489,51 1, filed January 21, 2000; a 
description of industry-standard color spaces may also be found in that application. The 
picture is then converted to a "standard" intermediate format, typically in one of the 
industry-standard color spaces. Examples include: 

L,a,b lobitsVpjxeL'channel (e,g, 5 used in Adobe Photoshop); 

SRGB Sbits/pixel/channel (e.g., used by Microsoft, HP, and others); and 

YUV 

The intermediate format is then mapped to the format required by the output 
device with the following processing: 

1 . Image scaling - to scale the image to the desired output size. 
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2. If only monochrome Mormatiori is desired, then a monochrome 
version of the image- is generated using standard conversion methods (e.g., using 
International Telecommtinicatioiii Union (ITU) recommendations for generating 
luminance signal Y from RjG^B signals, see, e.g.; ITU-Recommendation BT.6Q1- 

3 . If the bits/pixel is fewer than 8 - then dithering techniques (such as 
error diffusion, blue noise masking, of the like - see, e.g., Recent Progress in 
Digital Halftoning, 1994, The Society of Imaging Science and Technology, 
compiled and edited by Reiner- Eschbach, ISBN 0-892080-181-3). 

4. If the output device has a color palette (e.g., supporting only 256 
colors) then color dithering schemes are used. Such schemes are discussed hi 
some detail in Compute}' Graphics-Principles and Practice, Second Edition, 1990, 
Foley, van Dam, et aL, Addison-Wesley, Reading, MA, ISBN 0^201 -121 10-7. 

5. Compression is optionally performed before data is streamed out. Tor 
true-color images, the preferred compression scheme is JPEG. For indexed 
images GIF, PNG are the preferred methods. For halftoned images, the preferred 
approach is JBIG compression. 



20 Finally, the generated picture is outputted, and is ready for streaming to a target device 
for ultimate rendering on that device's display. 

An example of the operations of the media transformation module in reformatting 
an item of media content is shown by the following "AutoRotaieOp" function: 

25 Is #include "autorotateop.h" 

2 : 

3 s AutoRotateOp s : AutoRctateOp ( ) 

4= { 
5= } 
30 6 = 

7 : AutoRotateOp : : ~ft.utoROta.tepp. ( } 

6: { 

9= } 
10: 

35 1.1: const char * AutoRotateOp : .;.Name : U 

12: { 

13: return "autorotate* ; 
15: 

40 16; const char * AlitoSlOtateOp i 1 Arge { ) 

17: { 

16 : return "displayWidth(160J ■, displayHelght (120) , clockwise (I) " ; 
IB: } 

20: 
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void AutoRotateOp: : Process {lMS_itrtage *img) 
int32 w, h, cwj 

float display-Aspect, photoAspedt ; 

GetArg (&w, 160) ; 
Getflrgfslii 120); 
GetArg (Sew, 1) ; 

dieplayAspect = ( float) w / (float) h,- 

photoAopect = (float) (irog->widtb> / < float } < img - >he ight ) ; 

if i (dieplayAspect > l.Of && photoAepect < I. Of) |] 
(dlsplayAspect < l.Of pbotoAspect > l.Of)) 



40 i { 



if (cw) 

IMG_RotateRight (irag) ; 

a] se 

IMG__RatateLeft (img) ; 



The above AutoRotateOp function automatically rotates an image to better fit the 
30 available display of a particular device. As shown on line 26 above, the function, receives 
a. pointer to an image as a parameter. On lines 28 to 36, the display characteristics of the 
device are obtained. The condition on lines 38 to 39 evaluates whether or not the image 
should be presented in portrait orientation (i.e., to display the image vertically) or 
landscape orientation (i.e., to display The image horizontally) to better fit the device 
35 display. Depending on the outcome of this evaluation, a call is made to IMG RotateRight 
or IMG'RotateLeft as shown on lines 41 to 44 and the image is rotated either clockwise or 
counterclockwise to fit the display of a particular device. Source code listings providing 
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further details on the IMG RotateRight andlMGRotateLeft functions are attached hereto 
as Appendix A. 

The MTM uses cached versions of the original media obj ects whene ver po ssible. 
The MTM caches source media objects in the backside cache to reduce the time required 
to read the source media and to reduce Internet bandwidth consumption by the media 
delivery system. This backside caching can be performed by the MTM or by an 
intermediate reverse proxy cache. The- source objects are cached according to the 
directives specified in their HTTP cache-control headers. In the currently preferred 
embodiment, an Apache HTTP server configured as a reverse proxy cache is deployed 
"between" the Internet and the MTM module, so any requests for source multimedia 
documents are first compared to a local disk cache. The Apache reverse proxy cache 
module decouples the caching task from the media transformation task for better 
scalability and maintainability. 

Appended herewith is Computer Program Listing Appendix A containing, source 
listings, in the C/C++ programming language, providing further description of the present 
invention, A suitable environment for creating and building C/C++ programs is available 
from a variety of vendors, including Borland® C++ Builder available from Borland 
Software Corporation ofScotts Valley, California, and Microsoft® Visual C++ available 
from Microsoft Corporation of Redmond, WA. 

While the invention is described in some detail with specific reference to a single- 
preferred embodiment and certain alternatives, there is no intent to limit the invention to 
that particular embodiment or those specific alternatives, For instance, those skilled in 
the art will appreciate that modifications may be made to the preferred embodiment 
without departing from the teachings of the present invention. 
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COMPUTER PROGRAM LISTING APPENDIX A 



IMGLIB_API void IMG_PlipH(IMG_±B!age *img} 
^ if {i.Mg->pixglBytea != 4) 

^ DEBUS_IKG_FORMa.T; 

} return; 

inr.3 2 x, y, w; 
uintS *rowPfcr; 

uint32 *pixPtr, *pixPtr2, tempPix; 

k = img- ^vriath » 1 ; 
rowPtr = img->baseAddr ; 

,U (y _ 0; V < nq iL.'.L., / . t, 

pixPfcr = pixPtr2 = (uint32 *} rowPtr; 

+= img->width - 1; 
for {x = 0; X < W; X++) 

texpPix = "pixPCr; 
*pixPtr++ = *pixPtr2; 
■*pixPtr2-- = teiiipPix; 

rowPtr += img->rowBytes; 

} } 

IMGL.IBJ4.PI void IMG_FlipV(IMG_iraage *img) 

^ img->baseAddr +« img->rowBytes * (img->height - i) ,- 

^ img->rowBysee «■ -img->rowBytas; 

B1GIjJB_AP.I void IMG_Ro-ateLeft (IMG_image *itng) 
^ if Cimg->pixelBytea != 4} 



} 

IMG__image 



int.32 x, y; 

umt] i n r h r 

uint32 *pixPtr; 

if (itng->rowBytes -< 0) 

^ img->b.a.seAddr += <img->height - 1) * img->rowBytea; 

img- >rowByfce.s = -img->rowBytes ,- 

.lmg->width = templriig. height; 

img->lleight ™ tempimg. width; 

img - >rowByt es = ±mg->pixelBytes * img->width; 

rowPtr itrg->ta-aseMdr; 

s-uFu.v = pn _ i^E^Addr + teraplmg>pixeIByt«s * ( temp Irtig. width - 

for (y - C; y < img->height; y++j 

pixPtr = (uint32 *) rowPtr; 
srcPix = bxcRow; 

for (x = 0;x < img~>width; x*+) 

»pixPtr = * ( (uint32 *)srcPix} ; 



Page 32 of 51 



WO 03/0411893 



PCT/l)S02/3eo64 



pixKtr++; 

srcPk += tef^Irtg-rowBytes; 

rowPtr += Img- >rowBytcs ,- 
bxcRqw -= tentpImg.pixelByteS; 

^ IMG__Free Image (atemplmg) ; 

IKGEIB_API -void IMG_RotateRight (IMG_imsge *img 

^ if (iw\g->pixelBytes != 4) 

;DSBUG_IMG_FOSMAT ; 
ret: urn; 

} 

TMG_image temp Img; 



int32 x, y; 

uints *rowPtr, *srcRow, *srcPix; 
uint32 *?ixPtr; 

±f (i.mg->rowBytes < 0) 

^ img->baseAddr += [img->heigh.t - 1) * img ->rowBytes ; 

img->rowEytes = -img->rowBytes ; 

xmg->width = templmg , height; 

img~>height = templmg. widtji; 

itng->rowBytes = img- >pixel Bytes * img->width; 

rowPtr = xmg->baseAddr; 

srcRow = templmg . baseAddr 1 + temDlinq- , rowBvtee * (templnq. height 

1.) I 

for (y = 0? y < inig-sheight; y++) 

pixP^r o (uint32 *) rowPtr? 
srcPix = srcRow; 

fut- (x = 0; x < img->width; x++) 

*pixPtr = *< (;ui:nt32 *)srcPix); 
pixPtr-l-+; 

arcPix -= templmg. rowBytes; 

rowPtr += i mg-> rowBytes ; 
srcRow += templmg . plxelBytes ; 

^ IMGJFree Image (itcmplmc; ; 

IMGLIB_API void IMG_Rotatel80 (IMG image *img> 

^ IMG_FlipV(irag) ; 

^ IMGJFlipH(img) f 

1MGLI.B API void. IMG Resize {lMG_image *iirg, int32 newWidt.h, int32 
newHeight, bool higEQuality) 

if ( img ->pixel Bytes != 4} 



■ newWidth &i img->height == newHeight} 
lMG_iraage templmg;- 

if (IIMG^Allocatelmage^&tempImg, newWidth, newHeight, inn 
- -TJ-!- ' i J t) ) 

return; 



Page 33 of 51 



WO 03/0411893 



PCT/l)S02/3eo64 



//IMG StretchBlitCitng, atetnplmg, 0, 0, newWidth , aewHeight, true); 
"if (highQuality) 

Vlmage vlrag.; 
vlmg . Import (irog) ; 

vlmg. Scale [newWidth, newHeight, CImageServer : :CUB.IC) ; 
vlmg . Export ( Stemplmg) ; 



IHQ_StretfihBlit.(iing J Stemplmg, C, 0, newWidth, newHeight., 



lM'G_CopyTags (img, Stemplmg) ; 
*img = temping; 



else 
{ 



width - maxWidth; 
f - 

height = ROUND (f) ; 

height = CLAMP (height, 1, maxHeight) ; 



height = maxHeight; 

f =: (float) height *■ aspect; 

width.- CLAMP (width, 1, maxWidth); 



IMG He a i a e ( i mg , width , height , highQual ity) ; 



Apache i Module 

^ * This function creates a device capabilities object 

* and constructs the URL that is passed to the 

* Media Transcoding Module. 

* Sip.aram r 

* ©return 

static int uts_rewrite_uri (requeet_rec *r) { 
irtt Status ; 

// Gkip if already a proxy 
^ return OK; 

// skip all but GET requests 

if ; strcaaacmo : :r 3ET" , r->taei;hod) !- 0} i 

^ return OK; 

■■'/ get config file 

uts_dir config *c£g = (uts dir_canfig *): 

ap get_moduIe_conf ig (r- >per dir_confTg, &uts_moduie) ,- 

uts_server_config *scig = (uts server config *> 
ap_get_modul«_config (r-> server - >moduTe_conf Ty , &ut c module ) ; 

// (Bubrequest for 
strirg conht'itTvpe 
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* first, parse the -full Incoming DEI, We're going to treat the 
path+arqs 

* of this US.1 as the full URI for the proxy request 
uri__coraponentB uc; 

status = ap_paroe_uri components {r->pool, r->uri. Sue) ; 
if :-3-atus !=-HTTP OKT { 

ap_ log_ error (APIiOG MARK, APLOG_ERR | APLOGJSTOERRHO , r >serve-, 
"Bad input URI :■ %s", r->urT); 

return DECLINED; 

} 

char *path = ap_pstrdup(r->pool, uc.path); 

// pass in headers, get device capabilities back 
Devieeldentifier : : DeviceCap abilities devcap; 

status - dev_i5 t 11 1 Sdevoap, r, contentType) ; 

if {status ■! = DI SUCCESS) { 

ap log er:r-;irTA=T:OG__K^h!X, AP«0C_ERR [ APLOG_NOERKHO, r->server, 
"Device rap hj " it -s .u .4 fai - ' , 

ap. I09 -t i.^io -P._.DG__E?P I' APLOGJTOERRNO , r--> server, 

"Recording headers " 5 ; 

.logHeaders (r) ; 
return DECLINED ; 

} 

ap log_error {APLOG MARK, APT.0G_DEBDG | APL0G_NOERRNO , r --.-.server, 
•' t;r' , " okv cap LoiLiiis ' 

char *aub; 

// pop off initial slash ,, . . . , 

sub = ap_gatv.>ord_nc (r->poo_, Spath, '/')'; // initial slash 

// if we have an additional subscriber ID, pop it off, too 

if (c£g->3ubscrlber id_on_url) { 

^ sub = ap_getwora_nc(r->pool, Spath, '/'I; // subscriber ID 

'// Build, option list. 

// T0D0: This (Should really be a function that returns a string. 

char *cfg_clamp, *cf g_mimetype, *cfg path, *cf g_guali cy , 
*cfg_argpara.in, *cfg_rotate, *cfg_ecale, *cf g_f ilesise ; 
chat: b] sujk - '\0' ; 



ampsize=%d&" , 

hi .1 Lranscoderjiri, '?') != HULL ? «&" :. "?") , 

(devcap . display__height > devcap . display v/idth ? 
devc:a-D. display height •- h li It \ 1 1 

cfg_scale - ap_psprintf (r->pool, "%c", blank) ; 
} else { 

cfg clamp = ap psprintf (r->pool, "%s", 

13trc)i: j::o;-':.;ffic:a rcmscod.Rr 'iri, > I — NULL ? : "?") ) f 

cfg_scale = appsprintf (r- >pool, «l:im:i.tsize«%i,0&" , 
aevcap. display jtfidth) ; 

// Path 

if (path [st.rl.en (path) -2] ',') 
^ switch {path [strlen. (path.)" -1] } 

" a cfg_path =■ apjjsprint-f (r->pool, M in=\ "http : //%Bmp\"&" , path) 
break ? 

Ca clg_pafch ■- .ap_psprintf {r->pool t "i;n=\ "http : //%s..tfV'Sc n , path) 
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cfgrpath = ap_psprintf (r->pool, "in=\"Tife : tp://%spg\ l, &" , path) 



break ; 

cfgjath - ap_jispr±ntf <r->pool, "in=\"http://%si£\"£: : ", path) 
break f 
} else { 

efg_path = apjpsprintf {r->ppol, » in=\"http i//*b\ »■&•< , path) ,- 



// Mime Type 
c f g_mimetype - 
devcap . mime_type . c str ( ! 



pcol 



davc; cualicy: 



; ap_jpsprintf (r >pool, 



»outquality=%i:&". 
w outquaI.lfcy=%.l&" , 



// Rotate? 
• ( dsvcap , c:y:: Bi;:e_/:.o_j: :s t) 
cfg_rotate = ap psprintf (r->pool, 
devcap. diapl ay wi d th , 
devcap . dlsplsyjaeig&t) ; 

else 

cfg_rotate = ap_psprintf (r~>pool, 

// Arguments 
if (r->args) 

cfg_argparatn = 
else 

qfg_argparasi ■ 

// Filesize 
if (devcap; 
c£g_fi.i ler-i:^; 
devcap . maximum_f i le 



'autorotate=%i, fti, OS." , 



ap_pgprintf (r->pool, 
ap_psprintf (r->pool. 



' , r->args) 
blank) ; 



printf (r->pool, "f ilesise=%i& 
cfg_£ilesize = ap^psprintf (r->pooi, "%c", blank) 
list build 



/j^ End opti< 



// create URL for laps 

[1] URL [2] Clamp [3] path [4] format [b] quality [6] rotate [7] argparam [ 8 1 scale [9] f 
ilcsizc 

char *url = ap psprintf (r->pool, "%s%E%s%s$fs%sl-s%sfcs" , 
cfg- >media_transcoder_uri, 
cfg_clamp, 
cfg_path^ 
cfg_mirn^ , p - 
cf g_qual.ity, 
cf g_rote - 



APLC-G NOEKRNO , r->cerver, 



enetus » ap parse uri cor 
if (status' != HT*fP_OKT { 

ap_log error ( AP LOG MARK , APLOG_ERR 
"Bad proxy TJRI; %s", urij ; 



url ? sucj ; 
APLOGJSOERRNO, 



r-?servsr. 
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redirect won't work for remote servers, 

up as a proxy 
of working request member variables 
r = PROXY_PASS; 

• _= c ^pr 0 |y:http : // p6i 9l±~ 

tin=-ht£p : / /www . 1 ight surf . cow/ image s/misc/j 



r->proxyreq = PR0XY_PASS ; 

r ■• >ur.i = £p_petrd'ap {r->pool , uc.path) ; 

r- filename = ap_pstrcat (r-r>pool, "proxy:", uc. scheme, 
uc.hostinfo, uc.path, NULL) ? 

r->arga = ap_pstrdup ( r- >pocl , uc.guery; ; 



Devicaldentif ier 

^ * Deviceldentifier.cpp 
* implementation of Devieeldentifier and rteviceOapabilities classes 



#include <string> 
^include <strstream.li> 
#include <3tdlib.h> 



static Regex regex; 

Deviceldentif ier :: Devieeldentifier (coast char * filename) { 
try { 

config = new XMLConf igPi " - :i__u~ ij , 
catch (...) { 
} 1 

Devieeldentifier; : Devieeldentifier ( ) { 

^ config = new XMLConf igEile ( "./lsurf /uta/conf /devices .xml ") ; 



Deviceldenfc eler::-D* t eldeatifier () { 
if (config 1= NULL) { 
^ delete config; 



• K.etriftve a named capability value from the configuration 
■- f If Ih-U f { ' i m ,m 1 - j - 1 - i h. Ut ,-eapability> 
■ node identified by <cade>capabilitylfame</code:>, then 
■- determine, if any < header S.gt; tags apply. The 
' vali e returnee rs i string >hich can then be 
- converted to- the appropriate type. 
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* ©parara r 

* isparatn baseXPath 

* iBarBn i-H!pal'il :!.':v>!-".n» 

* ^return 

string Deviceldentif ier : :getCapability (req-aesfc_rec *r, string baseXPath, 

^* The device capabilities data storage is implemented as- an XML 

"* A member variable named "eonfig" allows us to make XPath queries 
* against the XML file. 

string cquery = 
bascxrach ■: 

sjtri.ii : capability l-Sn=.ns= \ " " : \- 
capabilitytlame + 
string ("\" 3 ") ; 

string value; 

// first, look through headers {if any) 

" ILJz in ':b -t^r - i — crType headernaiies - conf ig->query (cguery + 

tor. \ 
const char *headerval = ap_ table_get (r >headers_ j.is, 
headcrnam.es [i] .c_r.tr 0 5 ; 

if ilwa-tex-va". := ^ttjll) { 

// found a matching header, so grab the pattern and run the 

XMLGonf-igEile : ;StrihgVectorType patterne = oonfig- 
>query (cquery + string ' "/header [Sname=\ " " ! + headcrnames [i] +• 
■j.rlnq i " \ '' : /r:~-stterrr" ! .) ; 

if (regex. group (patterns [0] , 
// yes, there is a match 
and tile header value 

if (value. size () > 0) { 

II value now contains the matched substring 

break; 
} else { 

// there yjss a pattern match, but no parenthesized 
substring, so use the value of the "default" attribute 

XMLConfigTilei : StringVactorType defaults = eonfig- 
>query (cquery + string ( "/header [@name=\ "") + headernamee [i] + 
string ("\" 3 /©default") ) ; 

value = defaults [0] ; 

. ) 1 

> ' 

//if we didn't get anything from the headers, use the default value 
XEajCo^IbjFile - :3c.vingVectorType defaults = eonfig- >guery(equery + 
-A • 1 J tj i 11 . u ] t Cb nit 1 ; ; ; 

i.:: -ifaf-ults.ai.aer; > O'i { 
value = defaults {a] ; 

) 

II TODOt if there isn't a i 
// this may not be necessa 

file 



70 '* Convert the result of <code>getGapability 0 </code> 

* to an integer. 

* ©param r 

* Oparam baseXPath 

75 * ©param capabilityNarae 

* ©return 
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int Ceviceldentifier; :getCapatoilityInt{reg;uest_rec *r, string baseXPath, 

string value = getCapability (r, baseXPath, capabilityNauia) ; 
if (value. si ae{) > 0) { 

return atoi (value .c_str () ) ; 
} x . 

) 1 



* ©par am r 

* Oparam baseXPath 

s ^param capabi li tyN&me 

int Deviceldentif ier: :getCapabilityBool (request_rec *r, string 
baseXPath, string capabilityKame) { ... 
string value = getCapability ;r, base..':-.. capafcilltyW^) : 
return (lvalue. compare ("true") || ! value . compare ( " 1 " ) ) ; // anything 



* Fill in a DeviceCapabilites structure based on the 

* HTTP request headers and the content type of the. 

* requested document. The main job of this functioa 

* 1b ' to find the correct node in the config file for 

* the given User-Agent header, then call 

* <code>getCapabilItyStr{)</code>, <code>getCapabilityInt ( ) </code> , 

* and «code>getCapabilityBool()</code> to fill in 

* the device capabilities.* 

* Ipararr. devcap 

* ©param r 

* ©param contcntType 

Deviceldentif ier s igetDeviceCapabilities {Deviceldentif ier : :'DeviceCapabili 
ties *deycap , reguest_rec *r, string contentType) { 

if (devcap =- HULL) { 
// sorry, no can do 

ap_l oz - ;sr (7.M.C T-CIY., J»PjjOG_EER , j»server, "Cacnct call 
getDeviceCi3T.a"oil3 tie.s ') with a null DeviceCapabilitiss parameter"); 
return DI ERROR ; 

} 

-esulte? 



"User-Agent") ; 

xpath_results = conf ig->query ( "/ /user -agent/ ©pattern" } ; 
int found = FALSE; 

for (unsigned int i-Q; i < xpath_re suits ,siae {> ■< i++5 { 

if -regex. -ratch U-a-h resriiltF [.:.: ,, gtri xiy •, u« jta-erl : ) f 
base__T ' iv- H'//u»i -agent [@pattern=\" ") ; 

base_jxpath_guery . append (xpath results [i] } ; 
b=-at." A^dlh „ J„t / . attend 1 |: \ " j " : : 
b J J. = TRUE ; 

} else if' (regex, last error . size {) > 0) f 

// thio will complain a lot if: there are regex errors 
ap_log error (APLCG_MffiRK, MLOGJBRR, r->seirver , "Regular 
expression error; ¥rs , regex. last error. c_str O ) ; 
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ap log error (APLOGJKARK, APL0G_DEBU3, r->aerver, "Regex : 
%B», xpath_resultaTi] .c_str ()} / 

) ^ 

if < ! found) { 

ap log__error (AFLOG_MARK, APL0G__DESU3 , r->servsr, "Could not fxnd 
configuration entry for User -Agent \"%s\"", uajaeadsr) ; 
// TODO: log or send email 
re lu mi. D.. 

} 

// within that node, find the correct mime -type node 
xpath_re suits = c onf Ig- >query {base_xpath_t?iery + string {"/mime- 
type/@pattern") ) ; 

found = FALSE; , t . 

for {unsigned int i=0; i < xpa th_res-.il ts . Size 0 / [ 
if (r'egex. match (xpath_results [i] , contentType) ) { 

i ) ^f__- - *1 j t-_ : . -r 1 -> ' 1 'in-- - 1 1- u i 1 r-ir =\ "") ; 
base mat:. qi..<H.-y . suowl'A ■xfatM,_i'fi~r:Its :i.[ .! : 
base xpath_ query . append ( " \ " ] "T; 
founH = TRUE; 
break ; 

} else if ( regex.last_error. sise () > 0( { 
// again, here's , l i h ' - i i i plainer 

ud Icq error (A?LQG__KARK, API fx_.bil? , ..r-^efwr, 'T.agular 
expression error : , regex , ] asi^r-rror . r:_str|} ) ; 

!•:-<! e-'i'or AiA..,A:iJ<:A^ ; A- AAJ_lAB1A reserve.:, "Eoqvsx: 

%s", xpatb._resuit.BTi3 -c_str(}) ; 
if (i found) { 

ap_loq_er ror AAPLOG MARK, API,OG_DEBUG , r- /Eoiyci , "could not find 
<mime-type> entry in config file for document MIME type \"%b\ ,,b , 
contentType. c_str() ) ; 

// TODO: log or send email 
~~ NOTEC 



^ return DI_N0T FOUND 



arlOG debug, r->server, "Successfully 



// fill in the DeviceCapabilities structure 

// note defaults are: ••' 

devcap- >mime_type 
"output -mime -type" ) ; 

devcap->display_width = getCapabilitylnt {r, base_xpath_query, 

"display-width,") .; 

devcap->display_height = getCapabilitylnt {r, 1 

! 'dJ splay- height") , 

devcap->color_depth = getCapabilitylnt (r, base_xpath_query, 

11 color-depth" } ,- 

n.svCAp->i:i.^.ce_qi....H)lii:).- ■■- i: Cdpabi .. ity.i.nt (r, base_xpath query, 

"image- quality" } ; 

devcap->color__capablc = getCapabilityBool tr, base_xpath_query, 

"color-capable"}; 

devcap~>rotate_to__f it = getCapabilityBool (r, ba3e_xpath_query , 

"rotate-to-fit") ; 

dcvcai' icalej ._vidth = getCapabilityBool tr, base_xpath_query , 
"scale-to-width " J ; 

dsvcap->palette = getCapability (r r baa e_xpath_ que ry , 

"palette"} ; 

H~-c?p >maxim -_file_j3xze = getCapabilitylnt {r, base_xpath_query , 
"maximum- s ize " ) ; 

d c;:p->vid=o frane rata - getCapabili -y Act (r, har-3 xpa" !\_query , 

"video-frame-rate") ; 

de.vcap->audio_encoding_rate = -getCapabilitylnt (r, base_xpath__que.ry , 
"audio-encodinq rate"). ; 

devcap->audio_channels - getCapabilitylnt (r, base_xpath_guery, 

" i.''.idlo-t.;i-ai'!t::.elB " ) ; 
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// utility functions Co print out the DeviceCapab.ili.ties object 

void Deviceldentif ier i :DeviceCapabilities i rprint (o'stream *oa) { 
^ *ps « thiB->t.oString() ; 

string Device Identifier: :DeviceCapabilities : ;toString £3 { 
std: sostrstrsam buffer; 
buffer << "DeviceCapabilites {" 



mime -type; " « tnime_type 
display width .t " «. display _ width 



" ; -i ^ heiqht: : ■< < display height 
" ; color depth: >' « color depth 

«; color capable? " «. (color_capable ? "yes" : "no"; 

rotate to fit? " « (rotate_to_f it ? "yes" : "no"! 

video frame ratei 11 << v.. -:m-v..- invw. w:s 
"; audio- encoding rate: « « aaHio_ercodir.g rate 
■'; audio channelc: " «- audio_ channels 



}; 

ostreair. &op<sra.tar« {©stream &ob, Deviqeldentif ier : jDeviceCapabilitiee 

devcap. print (S.os) ,- 
return 6s; 

J; 
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WHAT IS CLAIMED IS: 

1 . In an online system, a method for determining the capabilities of client devices 
and supplying media content in a format suitable for such devices, the method 
comprising: 

receiving a request to provide a target device with a copy of a particular media 

object; 

determining capabilities of the target device; 

based on the capabilities of the target device, determining a format that is desired 
for providing the target device with a copy of the media object: 

translating the particular media object into a copy having said determined format; 
and providing the target device with the copy having said determined format. 

2. The method of claim 1 , former comprising 

storing the copy having said determined format in a cache memory. 

3. The method of claim 2, further comprising: 

receiving from a target device a subsequent request for the particular object in the 
determined format; and 

providing the target device with the copy stored in said cache memory. 

4. The method of claim 1 * further comprising: 

obtaining a copy of said particular media object from a connected server for 
translation of said media object, 

5 . The method of claim 4, further comprising: 

storing in cache memory a cached copy of said media object received from said 
connected server; and 

in response to subsequent requests for translation of said media object, using the 
copy of said media object stored in cache memory. 

6. The method of claim 1, wherein the capabilities of the target device include 
screen resolution. 
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7, The method of claim L wherein the capabilities of the target device include 
screen size. 

8. T i i ■ 11 i of claim I t irie h 'the i[ i il I ^ l i ^- <1 * c m ,i . 
color support. 

5 9. The method of claim I , wherein the capabilities of the target device include bit 

rate. 

10, The method of claim 1. wherein the capabilities of the target device include 
currently-available communication medium that the target device employs to transmit its 
request. 

10 11. The method of claim 10, wherein currently-av ailable communication 'medium 

comprises wireless communication. 

12. The method of claim 10, wherein currently-available cofxirnunication medium 
comprises wireline conrnlumcation. 

13. The method of claim 1, wherein said step of detetrrrmirtg capabilities of the 
1 5 target device includes examining the request submitted by the device 

14. The method of claim 1, wherein said step of determining capabilities of the 
target device includes examining the HTTP header submitted by the device. 

15. The method of claim 14, wherein examining the HTTP header submitted by 
the device, includes examining the HTTP User- Agent header. 

20 16. The method of claim 1 , wherein said step of determining capabilities of the 

target device includes querying the device for its capabilities. 

17. The method of claim 1, wherein said step of determining capabilities of the 
target, device includes determining capabilities from a knowledgebase, based on a device 
class for the target device, 

25 1 8. The method of claim 17, further comprising: 
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recording a log record, of target .d evices that are not recognized to enable the 
capabilities of said devices to be added to the knowledgebase. 

19. The method of claim 18, further comprising: 

automatically issuing notifications regarding said target devices that are not 

recognized. 

20. The method of claim 1, wherein said step of determining a format that is 
desired includes determining an appropriate resolution for rendering the particular image 
at the target device. 

21. The method of claim 1, wherein said step of determining a format that is 
desired includes determining an appropriate color space for rendering a particular image 
at the target device. 

22. The method of claim 1, wherein said step of determin ing a format that is 
desired includes determining an appropriate image size for rendering the particular image 
at the target device. 

23. The method of claim 1, wherein said step of determining a format that is 
desired includes determining whether to rotate the particular image to conform to the 
aspect ratio of the target device display. 

24. The method of claim 1, wherein said step of determining a format that is 
desired includes determining the appropriate bit rate- for the target device. 

25. The method of claim 1, wherein said step of determining a format that is 
de-sired includes determimn- ,. r , m bandwidth available for transmitting a copy 
of the particular media object to the target device. 

26. The method of claim 25, wherein the communication bandwidth available is 
determined, at least in part, based on the HTTP request header received from the target 
device. 

27. The method of claim 25, wherein the communication bandwidth available is 
determined, at least in part, based on a device class for the target device. 
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28, The method of claim 1, wherein said target device includes a handheld 
computing device having display capability. 

29. The method of claim 1, wherein, said target device includes a handheld 
computing device having digital: audio capability. 

5 30. The method of claim 1, wherein said target device includes a cellular phone 

device having display capability. 

31. The method of claim 1, wherein said target device includes a cellular phone 
device having digital audio capability. 

32. The method of claim 1, wherein said target device includes a pager device 
1 0 having display capability, 

33. The method of claim f, wherein said target device includes a personal 
computer having display capability. 

34. The method of claim 1, wherein said target device includes a personal 
computer having digital audio capability. 

15 35. The method of claim L wherein said target device includes WAP (Wireless 

Application Protocol) support. 

36. The method of claim 1, wherein said media objects include digital images. 

37. The method of claim 1, wherein said digital objects include digital video. 

38. The method of claim 1, wherein said digital objects include digital audio. 

20 39. An online system for providing digital media to target devices, the system 

comprising:' 

a capabilities module for dettTmiiihig the ^abilities of a particular target device; 
a transformation module for: 

automatically retrieving a oopy of a particular media object; and 



Page 45 of 51 



WO 03/040893 



PCT/US02/360M 



providing the: target device with a copy of said object, said copy being 
automatically translated into a particular format based on; the capabilities of the 
target device.. 

40. The system of claim 39, further comprising: 

5 a cache memory for storing copies of media objects that have been translated. 

41. The system of claim 40, wherein said system first attempts to satisfy the 
request by retrieving a copy of the particular object in the particular format from the 
cache memory. 

42. The system of claim 39, further comprising: 

10 a cache memory for storing copies of media objects that have been retrieved, 

43. Tile system of claim 4;-.. whersm said system first attempts to retrieve a copy 
of the particular media object from the cache memory before retrieving a copy from a 
remote server, 

44. The system of claim 39, wherein each digital object stored by said system is 
1 5 identified by a unique unifortn resource locator (URL) . 

45. The system of claim 44, wherein said unique URL is encoded with the 
characteristics of said digital object. 

46. The system of claim 44, wherein said unique URL includes the color depth of 
said digital object. 

20 47. The system of claim 44, wherein said unique URL includes the image size of 

said digital object. 

48, The system of claim 44, wherein said unique URL includes the resolution of 
said digital object, 

49. The system of claim 44, wherein said unique URL includes the bit rate of said 
25 digital object. 
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50. The system of claim 44, wherein said system stores URLs for each of the 
digital objects, and wherein the capabilities- module is capable of determining the 
particular digital object that may be provided to the target device from said URLs. 

51. The system of claim 39. wherein the capabilities of the target device include 
5 screen resolution. 

52. The system of claim 39, wherein the capabilities of the target device include 
screen size, 

53. The system of claim 39, wherein the capabilities of the target device include 
color support. 

10 54. The system of claim 39, wherein the capabilities of the target device include 

bit rate. 

55 . The system of cla'im 39, wherein the capabilities of the target device include 
currently-available communication medium that the target device employs to transmi t its 
request., 

15 56. The system of claim 55, wherein currently-available communication medium 

comprises wireless communication. 

57. The system of cl aim 55, wherein currently-available communication medium 
comprises wireline communication. 

58. The system of claim 39, wherein said capabilities module includes the ability 
20 to determine the capabilities of the target device from its HTTP header. 

59. The system of claim 58, wherein said capabilities module includes the ability 
to determine the capabilities of the target device- from its HTTP User-Agent header, 

60. The system of claim 39, wherein said capabilities module includes the ability 
to query the target device for its capabilities. 
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61. The system of claim 39, wherein said, capabilities module includes a 
knowledgebase for determining the capabilities of the target device based on its device 
class. 

.62. The system of claim 61, -further comprising: 
5 a log record for recording: target devices that are not recognized to enable the 

capabiliti es of said devices to be added to the knowledgebase. 

63. The system of claim 39, wherein said particular format is selected based on an 
appropriate resolution for rendering the particular media object at the target device. 

64. The system of claim 39, wherein said particular format: is selected based on an 
1 0 appropriate color space for rendering the particular media object at the target device. 

65 . The system of claim 39. wherein said particular format is selected based on an 
appropriate image size for rendering the particular media object at the target device, 

66. The system of claim 39, wherein said particular format is selected based on an 
appropriate bit rate for: rendering the particular media object at the target device. 

15 67. The system of claim 39, wherein said particul ar format is selected based on 

communication bandwidth available for transmitting a copy of the particular media object 
to She target device, 

68. The system of claim 67, wherein the communication bandwidth av ailable is 
determined, at least in part, based on a device class for the target device. 

20 69. The system of claim 39, wherein said target device includes a handheld 

computing device having display capability. 

70, The system of claim 39 t wherein- said target device includes a handheld device 
having digital audio capability, 

71. The system of claim 39, wherein said target device includes a cellular phone 
25 device having display capability. 
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72. The system of claim 39, wherein said target device includes a cellular phone 
device having digital audio capability. 

73, The system of claim 39, wherein said target device includes a pager device 
having display capability. 

5 74. The system of claim 39, wherein said target device includes a personal 

computer having display capability. 

75. The system of claim 39, wherein said target device includes a personal 
computer device having digital audio capability. 

76. The system of claim 39, wherein said target device includes WAP (Wireless 
10 Application Protocol) support. 

77. hi an online system, a method for determining the capabilities of client 
devices, the method comprising: 

receiving an original request from a target device in which said target device does 
not include information regarding its capabilities; 
15 determir 1 of the target device; 

supplementing said original request received from said target device with 
information about the capabilities of said target device; sad 

forwarding said supplemented request to a destination specified in said original 
request, 

20 78, The method of claim 77, wherein said step of determining capabilities of the 

target device includes examining the request submitted by the device 

79. The method of claim 77, wherein said step of determining capabilities of the 
target device includes examining the HTTP header submitted by the device. 

SO. The method of claim 79, wherein examining the HTTP header submitted by 
25 die device includes exarmmng the HTTP User- Agent header. 

81. The method, of claim 77, wherein said step of determining capabilities of the 
target device includes querying the device for its capabilities. 
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82. The method of claim 77, wherein said step of determining capabilities of the 
target, device includes detennining capabilities from a knowledgebase, based on a device 
class for the target device. 
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URLS FOR MULTIMEDIA OBJECTS IN WEB PAGES AT AN 
INTERNET SITE ARE ENCODED TO BE SERVED BY THE MEDIA 
DELIVERY SYSTEM 








402 


AN HTTP REQUEST FROM A CLIENT DEVICE IS ROUTED TO THE 
MEDIA DELIVERY SYSTEM 
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THE CCM RETRIEVES THE CLIENT CAPABILITY CONFIGURATION 
FROM THE DATA STORE USING THE HTTP HEADER AS A KEY 
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THE CCM CONSTRUCTS A URL CONTAINING INSTRUCTIONS FOR 
TRANSFORMATION OF THE SOURCE MEDIA OBJECT TO: 
CONFORM TO THE CAPABILITIES OF THE CLIENT DEVICE 
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THE CCM (OPTIONALLY} CONSULTS A CACHE FOR AN OBJECT 

MATCH iNG THE CONSTRUCTED URL AND, IF AVAILABLE, 
RETURNS IT TO THE CLIENT. IF A MATCHING OBJECT IS NOT 
IN. CACHE, THE METHOD PROCEEDS TO STEP 407 
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THE COM PROXIES THE ORIGINAL CLIENT REQUEST TO THE 
MTM, REPLACING THE URL SENT BY THE CLIENT WITH THE 
RECONSTRUCTED URL 



THE MTM RECEIVES THE RECONSTRUCTED URL AND MAKES AN 
HTTP REQUEST THROUGH A BACKSIDE CACHING SERVER FOR 
THE ORIGINAL MEDIA OBJECT. IF THE OBJECT IS NOT IN 
CACHE, THE CACHING SERVER REQUESTS IT FROM THE 
SERVER IDENTIFIED IN THE ORIGINAL CLIENT REQUEST 



THE MTM PERFORMS THE MEDIA TRANSFORMATION AS 
SPECIFIED IN RECONSTRUCTED URL 



THE NEWLY-TRANSFORMED VERSION OF THE MEDIA OBJECT IS 
RETURNED TO THE CLIENT DEVICE AND (OPTIONALLY) IS 
COPIED INTO CACHE 
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